Method for achieving end-to-end quality of service negotiations for distributed multi-media applications

ABSTRACT

This invention presents a framework for achieving dynamic End-to End QoS negotiation and control coordination, with distributed multimedia applications. The framework builds upon dynamic capability negotiations and specification of Adaptation Paths and (alternative) QoS Contracts, based on user preferences. In particular we present a protocol providing End-to-End negotiation of alternative QoS, capabilities, and preferences/configurations, based on extensions of IP-based protocols like SIP/RTSP/SDP, in coordination with mechanisms for network resource reservation (e.g. RSVP), local terminal resource (e.g. CPU, memory, power, auxiliary devices) reservation, and adaptation mechanisms. To this extent, and with respect to two or more peers ( 101, 103 ) this invention identifies six phases, through which said peers can establish multiparty, multi-stream, multimedia communications. In detail, the phases are: Protocol Discovery ( 104 ), Pre-Negotiation ( 106 ), Multi-Stream QoS Synchronization and QoS Correlation ( 107 ), Fast-Negotiation (obeying the Economy Principle) ( 108 ), Re-Negotiation (obeying the Economy Principle) ( 109 ), Resource Reservation Release ( 110 ). All the six phases can be concatenated, or be executed at different times. This invention also presents the concept of the E2ENP Broker ( 105 ), an optional third-party entity, which can be used for relieving peers ( 101, 103 ) from performing the time- and resource-consuming Pre-Negotiation phase ( 106 ) (and eventually also the Multi-Stream QoS Synchronization and QoS Correlation ( 107 ). This entity may coincide with e.g. audio-/videoconference bridges.

The invention hereby presented generally relates to the field ofdistributed multimedia applications and technologies. It includesresearch and development issues related especially to multimediamiddleware, Quality of Service (QoS) Management, Resource Reservationmechanisms, mobile terminals, and wireless communication.

At first the terminology used in the present description and the claimswill be shortly explained:

Term Definition Adaptation An ordered set of QoS contracts that indicatethe users Path preferences and wishes. Typically, the most important QoScontract is the first one in the path. Component A particular type ofinteraction (e.g. audio conversation) that can be realized usingdifferent applications Configuration A set of parameters required forimplementing a variation of a certain component. Potential configurationindicates the system's functional capabilities, whereas actualconfigurations reflect the mode of operation of a particularinstantiation. Content Exchange of information leading to select propernegotiation representation when transferring data Economy The economyprinciple suggests to reserve the more Principle expensive resources asthe last ones. As network resources are shared among several clients andtypically one has to pay for them, it is better to first reserveresources on all end systems and then resource network resources as thelast step. Feature set Some sets of media features described by a mediafeature assertion Quality of The collective effect of serviceperformance which Service, QoS determines the degree of satisfaction ofa user of the service¹ QoS Change The change of the QoS contractinitiated by the service user QoS Contract Agreement between a serviceuser and a given service provider, specifying QoS requirements andconstraints, as well as the policies required to keep track about QoSduring all phases of said service. QoS Contract Captures the structureof a class of QoS Contracts, by Type identifying how individual QoSContracts specify the QoS properties over a given set of QoS parametertypes (also known as dimensions in [Frolu98]). Each parameter typeconsists of a name and a domain of values. QoS specifications can besimply intended as a set of constraints over said domains, one perparameter type. QoS General term for identifying set of QoS parametersand Specification constraints specified by a user. QoS Violation Theviolation of a QoS Contract caused by the Service Provider Third partycall In third party call control, an external controller is controlcreating, modifying and terminating sessions between other participants.¹Definition from the ITU-T (former CCITT) Recommendation E.800

Now the abbreviations used in the description will be explained:

ATM Asynchronous Transfer Mode CC/PP Composite Capabilities/PreferenceProfiles CSCW Computer-Supported Cooperative Work E2ENP End-to-EndNegotiation Protocol HDTV High Definition Television HTTP Hyper TextTransport Protocol IETF Internet Engineering Task Force IP InternetProtocol IRT Initiator-Role-Token MSC Message Sequence Charts OSOperating System RDF Resource Description Framework RTP Real TimeProtocol RTSP Real Time Streaming Protocol RSVP Resource ReservationProtocol SAP Session Announcement Protocol SDP Session DescriptionProtocol SIP Session Initiation Protocol VoD Video on Demand UML UnifiedModeling Language XML Extended Markup Language

The following prior art documents are known to the inventors:

-   [Beser00] B. Beser, Codec Capabilities Attribute for SDP, IETF    Internet-Draft, Work-in-progress,    <draft-beser-mmusic-capabilities-00.txt>.-   [Bhatt99] S. Bhatti, G. Knight, Enabling QoS adaptation decisions    for Internet applications, London, UK, 1999.-   [Blake98] S. Blake, D. Black, M. Carlson, E. Davies, Z. Wang, W.    Weiss, An Architecture for Differentiated Services, IETF Request for    Comments: 2475, December 1998.-   [Booch99] G. Booch, J. Rumbaugh, I. Jacobson, The Unified Modeling    Language User Guide, Addison Wesley Longman, 1999.-   [Bor00] C. Bormann et. al., Simple Conference Control Protocol, IETF    Internet-Draft, Work-in-progress, <draft-ietf-mmusic-sccp-01.txt>.-   [Burn00] L. Burness et al., The BRAIN Quality of Service    Architecture for Adaptable Services with Mobility Support, in    Proceedings of the PIMRC 2000, London, 2000.-   [Cama00] G. Camarillo, Confirmation of SDP preconditions, IETF    Internet Draft, Work-in-progress,    <draft-camarillo-manyfolks-confirm-02.txt>.-   [Cama00a] G. Camarillo, Third party call control with SDP    preconditions, IETF Internet Draft, Work-in-progress,    <draft-camarillo-3pcc-qos-00.txt>.-   [Conneg00] G. Klyne (ed.), A revided media feature set matching    algorithm, IETF media feature registration WG, Work-in-progress,    <draft-klyne-conneg-feature-match-02.txt>.-   [Conneg00a] G. Klyne (ed.), Identifying composite media features,    IETF conneg working group, Work-in-progress,    <draft-ietf-conneg-feature-hash-05.txt>.-   [Frolu98] S. Frolund, J. Koistinien, QML: A Language for Quality of    Service Specification, HP-Lab Technical Reports, HPL-98-10, 980210.-   [Handl98] M. Handley, V. Jacobson, SDP: Session Description    Protocol, IETF Request for Comments: 2327, April 1998.-   [Handl99]M. Handley, H. Schulzrinne, E. Schooler, J. Rosenberg, SIP:    Session Initiation Protocol, IETF Request for Comments: 2543, March    1999.-   [Handl00] M. Handley, C. Perkins, E. Whelan, Session Announcement    Protocol, IETF Request for Comments: 2974, October 2000.-   [ISOX641] ITU-T Recommendation X.641 (December/1997) |ISO/IEC    13236:1998, Information technology—Quality of Service: Framework.-   [Kassl99a] A. Kassler, P. Schulthess, An End-to-End Quality of    Service Management Architecture for Wireless ATM networks, in:    Proceedings of the HICSS 32, Mauii, Hi., USA, January 1999.-   [Kassl99b] A. Kassler, P. Schulthess, Extending the QoS Paradigm of    Wireless ATM to Mobile Broadband Applications and to the User, in:    Proceedings of the WCNC '99, September 1999, New Orleans, USA.-   [Kassl00] A. Kassler et al., BRENTA—Supporting Mobility and Quality    of Service for Adaptable Multimedia Communication, in Proceedings of    the IST Mobile Communications Summit 2000, Galway, Ireland, October    2000, pp. 403-408.-   [Kassl00a] A. Kassler et al., An Open Endsystem Architecture for    Adaptable Multimedia Services with QoS Support, in Proceedings of    the BRAIN workshop, London, 2000.-   [Levin01] O. Levin, SIP Requirements for support of Multimedia and    Video, IETF Internet-Draft, Work-in-progress,    <draft-levin-sip-for-video-00.txt>-   [Manda00a] D. Mandato, A universal QoS adaptation framework for    mobile multimedia applications, European Patent Application 00 111    191.3, May 24, 2000.-   [Manda00b] D. Mandato et at, High-Level interface for QoS-based    mobile multimedia applications, European Patent Application 00 126    975.2, Dec. 8, 2000.-   [Marsh00] W. Marshall et al., SIP Extensions for Resource    Management, IETF draft <draft-ietf-sip-manyfolks-resource-00>,    November 2000.-   [Nahr95] K. Nahrstedt and J. M. Smith, The QoS Broker IEEE,    Multimedia Magazine, Spring 1995 (2)1, pp. 53-67-   [Ott99] J. Ott et. al., Capability description for group    cooperation, IFTF Internet-Draft, Work-in-progress,    <draft-ott-mmusic-cap-00.txt>.-   [Pan99] P. Pan, H. Schulzrinne, YESSIR: A Simple Reservation    Mechanism for the Internet, Computer Communications Review (CCR),    Vol. 29, No. 2, pp. 89-101, April 1999.-   [Pasqu92] J. Pasquale, G. Polyzos, E. Anderson, V. Kompella, The    Multimedia Multicast Channel, Proc. of 3rd International Workshop on    Network and Operating System Support for Digital Audio and Video    (NOSSDAV 92), San Diego, Calif., November 1992, pp. 185-192.-   [Pasqu93] J. Pasquale et al., Filter Propagation in Dissemination    Trees: Trading Off Bandwidth and Processing in Continuous Media    Networks, Proc. of NOSSDAV 93, Lancaster University, Lancaster, UK,    1993, pp. 269-278.-   [RFC2533] RFC 2533, A Syntax for Describing Media Feature Sets,    Graham Klyne, 5 GM/Content Technologies, March 1999.-   [RFC2703] RFC 2703, Protocol-independent Content Negotiation    Framework, Graham Klyne, 5 GM/Content Technologies, September 1999.-   [SIPRES01] W. Marshall et. al., Integration of Resource Management    and SIP—SIP Extensions for Resource Management, IETF SIP working    group, Work-in-progress, <draft-ietf-sip-manyfolks-resource-01.txt>.-   (SDPCN00) F. Andreasen, SDP Simple Capability Negotiation, IETF    MMUSIC working group, Work-in-progress,    <draft-andreasen-mmusic-sdp-simcap-00.txt>.-   [SDPCN01] F. Andreasen, SDP Simple Capability Negotiation    Requirements, IETF MMUSIC working group, Work-in-progress,    <draft-andreasen-mmusic-sdp-simcap-reqts-00.txt>.-   [SDPNG00] D. Kutscher et. al., Requirements for Session Description    and Capability Negotiation, IETF Internet-Draft, Work-in-progress,    <draft-kutscher-mmusic-sdpng-req-01—.txt>.-   [Shulz98] H. Schulzrinne, A. Rao, R. Lanphier, “Real Time Streaming    Protocol (RTSP)”, IETF Request for Comments: 2326, April 1998.-   [SIPRES01] W. Marshall et. al., Integration of Resource Management    and SIP—SIP Extensions for Resource Management, IETF SIP working    group, Work-in-progress, <draft-ietf-sip-manyfolks-resource-01.txt>.-   [SMIL98] Synchronized Multimedia Integration Language (SMIL) 1.0    Specification, W3C Recommendation, Jun. 15, 1998.-   [Yeado93] N. Yeadon, F. Garcia, D. Hutchinson, D. Shepherd, Filters:    QoS Support Mechanisms for Multipeer Communications, IEEE Journal on    Selected Areas in Communications, Vol. 14, No., 7, September 1993.

The relevant prior art documents will now be acknowledged shortly:

In W. Marshall et. al.: Integration of Resource Management and SIP-SIPExtensions for Resource Management, IETF SIP working group,Work-in-progress, <draft-ietf-sip-manyfolks-resource-01.txt>, theauthors present a multi-phase call-setup mechanism that makes networkQoS and security establishment a precondition to sessions initiated bythe Session Initiation Protocol (SIP) and described by the SessionDescription Protocol (SDP). Network resources are reserved before thesession is started using existing network resource reservationmechanisms (like RSVP). The resource management protocol is interleavedbetween two phases of call signaling and participants are invited afterresources are available in the network. A confirmation of thepreconditions is explicitly signaled. Resource management is done onlyfor the network resources. An extension to SDP is introduced todetermine whether the preconditions are met.

In G. Camarillo: Confirmation of SDP preconditions, IETF Internet Draft,Work-in-progress, <draft-camarillo-manyfolks-confirm-02.txt>, anadditional direction attribute is introduced to indicate which partysends the confirmation of the preconditions. Finally, G. Camarillo:Confirmation of SDP preconditions, IETF Internet Draft,Work-in-progress, <draft-camarillo-manyfolks-confirm-02.txt>provides amechanism to perform third party call control in SIP when SDPpreconditions are used.

In RFC 2533, A Syntax for Describing Media Feature Sets, Graham Klyne,5GM/Content Technologies, March 1999 the authors present a format toexpress media feature sets that represent media handling capabilities.In addition, an algorithm is provided that matches the feature sets. Itmight be used to determine if the capabilities of a sender and receiverare compatible.

This matching algorithm is improved in G. Klyne (ed.): A revised mediafeature set matching algorithm, IETF media feature registration WG,Work-in-progress, <draft-klyne-conneg-feature-match-02.txt>.

In addition, G. Klyne (ed.): Identifying composite media features, IETFconneg working group, Work-in-progress,<draft-ietf-conneg-feature-hash-05.txt> describes an abbreviated formatfor composite media feature sets that uses a hash of the featurerepresentation to describe the composite. This can be used to provide anabbreviated form for referencing an arbitrary feature set expression,independent of any particular mechanism for dereferencing. In F.Andreasen, SDP Simple Capability Negotiation Requirements, IETF MMUSICworking group, Work-in-progress,<draft-andreasen-mmusic-sdp-simcap-reqts-00.txt> the authors state therequirement that a capability set should contain a handle (similar tothe hash mentioned above) allowing for easy referencing of thecapability set.

In RFC 2703, Protocol-independent Content Negotiation Framework, GrahamKlyne, 5 GM/Content Technologies, September 1999 the authors present anabstract framework for protocol independent content negotiation for theresources with which it interacts. It does however not provide thecontent negotiation process. It identifies the need for expressing thecapabilities of the sender and the data resource to be transmitted, thecapabilities of the receiver and the need for a protocol to exchangethese capabilities. Negotiation is carried out by a series ofnegotiation metadata exchanges. Negotiation stops, if a specific datafile has been found to be transmitted. The sender transfers the data ifthe sender determines the file, otherwise the receiver informs thesender. This proposal therefore relates to content negotiation.

Work going on in D. Kutscher et. al.: Requirements for SessionDescription and Capability Negotiation, IETF Internet-Draft,Work-in-progress, <draft-kutscher-mmusic-sdpng-req-01.txt> identifiesthe requirements of a framework dealing with session description andendpoint capability negotiation in multiparty multimedia conferencingscenarios. Depending on user preferences, system capabilities or otherconstraints, different configurations can be chosen for the conference.The need of a negotiation process among the peers is identified, but notdescribed, in order to determine a common set of potentialconfigurations and to select one of the common configurations to be usedfor information exchange. This capability negotiation is used to get avalid session description compatible with the end system capabilitiesand user preferences of the potential participants. Differentnegotiation policies may be used to reflect different conference types.They also identify the need for network resource reservation coupledwith session setup. Finally a proposal is drafted for describingcapabilities and providing the negotiation language but not a protocol.

The authors of F. Andreasen, SDP Simple Capability Negotiation, IETFMMUSIC working group, Work-in-progress,<draft-andreasen-mmusic-sdp-simcap-00.txt> propose a set of SDPextensions providing a minimal and backward compatible capabilitynegotiation mechanism. In B. Beser: Codec Capabilities Attribute forSDP, IETF Internet-Draft, Work-in-progress,<draft-beser-mmusic-capabilities-00.txt> the author extends SDP so thatend-points know the codec choices and can agree on a common set. Thecommunication partner can thus obtain the originators capabilities andpreferences. In J. Ott et. al.: Capability description for groupcooperation, IETF Internet-Draft, Work-in-progress,<draft-ott-mmusic-cap-00.txt> a notation for describing potential andspecific configurations of end systems in multiparty collaborationsessions is given. This enables mechanisms to define end systemcapabilities, calculate a set of common capabilities and to express aselected media description for use in session descriptions. They do notprovide a protocol for capability exchange. In C. Bormann et. al.:Simple Conference Control Protocol, IETF Internet-Draft,Work-in-progress, <draft-ietf-mmusic-sccp-01.txt> the authors defineservices for a simple conference control protocol (SCCP) for tightlycoupled conferences. Member management, application/session managementand access control rules for distributed application resources aredefined. The conference's state, which might be established using SIP,is managed during the lifetime using SCCP. This includes the finding ofappropriate configurations, negotiating for configurations and changingthe configuration. However, no interaction with local resourcemanagement is intended.

The authors of K. Nahrstedt and J. M. Smith: The QoS Broker, IEEEMultimedia Magazine, Spring 1995 (2)1, pp. 53-67 presented a model foran end-point architecture based on a QoS Broker, which is a functionalentity that orchestrates resources at the end-points and coordinatesresource management across layers. In order to configure the systemproperly, the broker uses admission control and negotiation. Negotiationamong peer entities leads to a valid configuration, which involves allnecessary components of the communication system.

In the present document the term economy principle is used to describethe order of reservation described in K. Nahrstedt and J. M. Smith: TheQoS Broker, IEEE Multimedia Magazine, Spring 1995 (2)1, pp. 53-67:

-   -   First, local resources are reserved.    -   Then negotiation with the peer entity leads to a configuration        that can be mapped to resource requirements at the peer, which        are then reserved.    -   Finally, reservation of network resources is done in the last        step, because network resources are expensive and shared among        multiple users.

The authors of O. Levin, SIP Requirements for support of Multimedia andVideo, IETF Internet-Draft, Work-in-progress,<draft-levin-sip-for-video-00.txt> present a set of requirements for acall-control protocol for real-time multimedia support over IP.Capabilities have to be expressed, capabilities have to be signaled toidentify the amount of resources that are necessary, and a call controlmechanisms is needed to open/close/modify media streams within theboundaries set forth by capabilities and reserved resources. Also theypropose to announce new capabilities (if available) during a session. Inaddition, the peers have to agree on a common set of codecs to be used.A session control mechanism to start/stop the streams is put as arequirement.

In Burness L. et al., The BRAIN Quality of Service Architecture forAdaptable Services with Mobility Support, in Proceedings of the PIMRC2000, London, 2000; Kassler A. et al., BRENTA—Supporting Mobility andQuality of Service for Adaptable Multimedia Communication, inProceedings of the IST Mobile Communications Summit 2000, Galway,Ireland, October 2000, pp. 403-408; and Kassler A. et al., An OpenEndsystem Architecture for Adaptable Multimedia Services with QoSSupport, in Proceedings of the BRAIN workshop, London, 2000, anend-system architecture is presented that integrates local, peer andnetwork resource reservation into a framework for end-to-end QoSmanagement. User Preferences and adaptation paths are used together withQoS states to negotiate QoS at application level. Interaction with localresource management is introduced. The layered architecture providessupport for different types of applications.

The basic idea of extending existing session protocol layers, like SIP,for pre-negotiating Adaptation Paths at multiple levels of streamaggregations, is contained in European Patent Application 00 126 975.2,Dec. 8, 2000. However, this European patent application does not detaila full-fledged End-to-End QoS Negotiation Protocol, and does notintegrate admission control and resource reservation aspects.

In the European Patent Application 00 111 191.3, May 24, 2000, a datamodel detailing ways to achieve QoS synchronization and QoS correlationamong multiple multimedia streams is presented. Such data model refinesthe concepts presented in the aforementioned European Patent Application00 126 975.2. Such a data model, however, does not cover the descriptionof capability like codecs.

PROBLEM TO BE SOLVED BY THE INVENTION

In [Levin01], the author states on page 2: “(. . .) establishing asession of a certain quality, based on capabilities agreement during thesession establishment and sustained throughout the duration of thesession. The problem can be described as a lack of expressiveness inthree following areas: capabilities specification, resource reservationand media stream control.

The desired ultimate goal is

-   -   to express the capabilities (i.e. supported media, CODEC        algorithms, bandwidth, etc.) without a need in configuration,    -   to signal the total resources required for a specific session        (probably in terms of capabilities, and    -   within a session, to explicitly open and close media streams and        to modify the parameters of a certain stream within the        boundaries of previously announced capabilities and reserved        resources.”

The author also states on page 4: “SIP extensions may be needed toseparate the ‘capabilities announcement’ phase from the actual openingof media streams.”

In view of the prior art it is the objective of the present invention topropose a concept providing QoS guarantees in an effective manner.

This objective is achieved my means of the features of the independentclaims. The dependent claims develop further the central idea of thepresent invention.

Further features, objects and advantages of the present invention willnow be explained with reference to the figures of the enclosed drawings:

FIG. 1 shows high-level MSC identifying phases,

FIG. 2 shows a functional decomposition of a structure to implement thepresent invention,

FIG. 3 shows an example of a pre-negotiation phase 106 (MSC),

FIG. 4 shows an example of a fast negotiation phase 108 (MSC), and

FIG. 5 shows an example for the implementation of an E2ENP Broker incase of SIP implementation.

Following are those requirements which can be of importance forproviding QoS guarantees in an effective manner:

-   -   Provision of a comprehensive set of mechanisms and protocols,        linking together several aspects like:        -   1. Pre-Negotiation, which also can be done offline and            published using e.g. LDAP        -   2. Negotiation integrated with ad-hoc reservation        -   2.1 Economy principle        -   2.2 Coordination of resource reservation, enhanced with the            features listed below    -   Specific description of the interaction with local and peer        resource management (some protocol procedures are required to        this extent)    -   Use of state IDs to construct and reference an Adaptation Path    -   Explicit mention to hierarchical state machines for Adaptation        Path at different levels

One of the targets the invention is aiming at is the support of varioustypes of services:

-   -   Conversational services    -   Distribution services    -   Information retrieval services

The concept proposed by the present invention can be adapted to thesedifferent types of services. The reason being that, depending on thetype of service, different sequences of actions may be required for theinteraction between reserving network and end system resources, whichalso depends on the communication mode of each peer (sender/receiver).

For instance, questions need to be addressed, like whether a receiverneeds to wait to be contacted by a sender, or who initiates resourcereservation on the network (the sender or the receiver). The answers tothese questions depend on the type of service.

The Concept of an End-to-End Negotiation Protocol (E2ENP)

The establishment of a QoS-enabled communication session can beaccomplished in a multi-step process, starting with negotiation of QoSaspects on an end-to-end basis. The idea hereby proposed is to havepeers negotiating beforehand (i.e. before the actual communication takesplace) a common level of QoS, the use of which peers can agree upon.This is a form of pre-negotiation of a vocabulary which peers can uselater on for efficiently dealing with contingent negotiations (e.g. whenestablishing audio/video streams) or re-negotiations (changing the QoSContract during ongoing sessions). The benefit of this approach is thatthe time necessary for re-negotiation is reduced because the peers haveonly to refer to a negotiated state instead of performing a fullnegotiation cycle during streaming. Furthermore, the type of negotiationcan be tailored based on capabilities that are currently available atall peers.

The identified steps are:

-   1. Pre-negotiation of a set of QoS contracts with the peers. Such    set describes an Adaptation Path for a generic stream of a given    type (e.g. audio, video, or data streams).-   2. As an intermediate step, peers negotiate QoS correlation and    synchronization aspects among multiple streams. This might not be    necessary if a session contains only one stream: in that case QoS    correlation would be simply ignored. Thus this step builds up the    session concept. Note that, in case peers have to coordinate several    streams, QoS correlation and synchronization issues would be handled    at runtime by triggering specific coordination function (based on    e.g. RTP), which are derived from the negotiated QoS states.-   3. As a last step, peers negotiate QoS Contracts on a per stream    basis at stream establishment time, based on the pre-negotiated    vocabulary established during step (1). At this step, actual    admission control is performed. To this extent, according to one    aspect of the present invention it is proposed to follow an economy    principle observing the fact that network resources have to be    shared and are more expensive than local resources. The hereby    proposed process is based on the steps in the following order:    -   3.1 Local resources (i.e. resources managed by the local system)        like CPU, memory, and power are reserved on the terminal device        of the party initiating the communication (i.e. the initiator).        These are the cheapest resources, since they are used        exclusively by the initiator. The amount of resources that are        reserved by the initiating party corresponds to the quality the        initiator is most interested in and can cope with.    -   3.2 Remote resources like CPU, memory, and power are reserved on        the terminal device of the party accepting the request to        establish the communication (i.e. the responder). These        resources may be eventually shared by a plurality of initiators        with respect to multiple telecommunication sessions, like in the        case of a Video-on-Demand scenario (where the responder is the        video server, which needs to serve multiple requests from        multiple users). It does not make sense to try to reserve these        shared resources (thus affecting other communication sessions at        the responder) prior of having ascertained the availability of        resources at the initiator side.    -   3.3 Only when both the local and remote resources have been        successfully reserved, network resources are reserved which are        shared (by definition) by a plurality of parties and therefore        to this extent can be considered as the most expensive resources        at all.-   4. This invention describes a protocol procedure for informing peers    about changes in capability configuration (e.g. the installation or    removal of a given multimedia codec) of a given participant of the    given communication session. By pro-actively disseminating this    information among peers, proper actions can be taken with respect to    any currently active communication session, and to all the    negotiation processes concerning future    attempts-to-establish-communication sessions.

Steps (2) and (3) (or (1)², (2), and (3)) can be treated jointly asone-shot³ or separately as a sequence of actions⁴. ² In such a case,pre-negotiations might be no longer valid, should a new party join asession with a different understanding what an adaptation path is withrespect to the already negotiated on.³ As an example, negotiation andset up of a session with two audio- and three video-streams,simultaneously⁴ Carry out the pre-negotiation first, then start with thefirst stream, add then the second stream, and so on.

Note that in certain cases only steps (3.1), (3.2) and (3.3) might benecessary. As an example this can be the case if everything ispre-negotiated by default (using fixed settings) and only one qualityand configuration of a video stream is located on a server.

A key aspect of the E2ENP is that it can be used not only in conjunctionwith network resource reservation signaling protocols, but also withother types of QoS architectures. In the latter case, pre-negotiationwill focus on having peers agreeing on the types of QoS classes to use,with respect to the capabilities of each peer.

What to Negotiate

This section focuses on (i) what type of information (ii) under whichcircumstances needs to be negotiated.

More specifically, peers should agree on:

-   -   which E2ENP (or other protocol) to use;    -   which capabilities and configurations are available and        applicable;    -   what QoS contracts are applicable and form the Adaptation Path;        and    -   the Adaptation Paths themselves.        Types of Supported E2ENP

Most of the state of the art applications will not be able to employ thehereby-described E2ENP without any specific retrofitting. Rather theywill eventually apply some other form of state of art negotiationmechanism. It is reasonable to assume also that some future applicationsmay eventually only partially support the E2ENP, or just differentversions thereof.

For these reasons, the first type of information that needs to benegotiated is the type and version of supported “E2ENP”.

This might be accomplished by using a directory server. As an example, adirectory entry associated with a given peer, and named E2ENP would actas a token indicating that the given peer supports the E2ENP. Thedirectory entry could look like:

where “type of service” could indicate information retrieval service orconversational service, “protocol” could indicate SIP. (together withSDP) or H.323 or RTSP (i.e. which protocol is the given E2ENP basedupon), and “version” could be a number discriminating incremental setsof functionality. In addition to the aforementioned protocols, specificextensions are needed to carry out the negotiation and describe theAdaptation Path.

As an example, the E2ENP can be mapped—based on the type of service—tothe following protocols:

Type of Service Protocols Conversational services SIP(together withSDP)/H.323 + enhancements Distribution services SAP + RTSP +enhancements Audio/Video on Demand RTSP + enhancements (Push/Pull)services

This information can be not only directly queried on purpose by thepeers, but the directory servers themselves can also pro-activelydisseminate it.

Capabilities

Various types of capabilities need to be negotiated among peers:

-   -   Media Types and QoS Contract Dimensions    -   Bitrates    -   Codecs    -   Terminal capabilities    -   Network Resource Reservation support

The following paragraphs analyze in detail each type of capabilities.

Media Types and QoS Contract Dimensions

The type of media (audio/video/data) can be used for screening any pieceof information in the E2ENP messages. Peers agree on a common set ofmedia types. As an example, if a receiver requests to establish a liveaudio stream but the corresponding audio stream sender does not have amicrophone, further negotiation of audio codecs is pointless andtherefore it can be skipped. The use of media types thus translates instructuring the E2ENP messages in a format which is convenient for thepeers while negotiating. In addition to media types, this inventionidentifies the need to include the list of QoS Contract Dimensions whichthe contracts are made of. As an example, one peer might want tonegotiate frame-rate and frame-size, whereas other peers may want tonegotiate frame-rate only.

As an example, it might be useless to try to negotiate frame-size, if aVoD server has only one representation (version) of the movie available,and there is no network media adaptation unit available.

This invention proposes the following format describing media typeinformation:

The refinements “param_audio” and “param_video” detail the QoS contractdimensions, like frame-size, frame-rate for video and sampling frequencyand number of channels for audio, and are described in the nextsections. This format is to be applied to each media type description ina given E2ENP message.

Codecs

Before peers can negotiate QoS Contracts or even Adaptation Paths (seebelow), they shall agree first on which codecs are generally applicable.The QoS achievable depends also on the type of codecs available.Furthermore, even though a given peer might be equipped with a given setof codecs, the use of such codecs is actually constrained by theavailability of equivalent codecs by all the other peers.

Today's SIP implementations allow the negotiation of codecs atinvitation time on a peer-to-peer basis, and with respect to thespecification of well-defined streams. In the framework of the presentinvention an extension of such a SIP capability is proposed, bynegotiating during step (1) of the above cited steps (1) to (4) thecomplete list of available codecs, irrespective of which streams will beused later at stream establishment time. In other words, peers can inthis way agree on a common subset of codecs before the actual mediaspecification takes place.

Furthermore, since current and future terminal devices shall be able toinstall or remove codecs “on the fly”, the common set of codecs mightneed to be later re-negotiated during connection time (e.g. during avideoconference). The re-negotiation could be achieved by using new SIPmessages.

Bitrates

Two main mechanisms are employed by clients to minimize the amount ofbandwidth used for multimedia applications:

-   -   Using low (constant) bitrate codec thus reducing network        resource consumption,    -   Using variable bitrate codecs

Clients have to agree on a certain set of compatible codecs. It must,however, be assured that the bitrate the codec generates is supportedboth by the clients (for sending/receiving) and by the network (both theaccess networks and the core). In addition, one codec may generatedifferent bitrate levels, which depend on the codec configuration. As anexample, the G.726 codec may generate bitrates of 16, 24, 23 or 40 kb/s.There may be situations where the receiver is able to handle only acertain configuration of the given codec. As an example, if the receiveruses a 19.6 kb/s modem, the receiver can only handle the 16 kb/sconfiguration of the G.726 codec. In addition, certain audio codecs andseveral video codecs can generate a variable bitrate, which is typicallyspecified using a leaky bucket model. In this case the bitraterequirements are specified using a sustainable bitrate, a peak bitrateand a burst time (time duration that indicates how long the source isallowed to send at the peak rate).

This invention proposes that in addition to a negotiation of the set ofcompatible codecs, for each codec the peers have to agree on a validcodec configuration in terms of bandwidth that each peer can handle. Theinitiator provides and negotiates a set of bandwidth specifications withthe peers using the leaky bucket model (sustainable bitrate, peakbitrate, burst time), which supports both constant and variablebitrates. In the above example, both parties would agree on the G.726codec with the following bitrate specification: (sustainable bitrate,peak bitrate, burst time)=(16, 16, N/A).

Having agreed on a compatible codec configuration in terms of bitrates,actual target bitrates that are being used in the actual configurationare negotiated with the network during step (3).

Terminal Capabilities

The physical characteristics (e.g. monitor size) of terminal devices maychange from is model to model, and affect the delivery of services overtelecommunication networks. This is important, as it makes no sense forinstance to stream a video in HDTV quality to a mobile phone, which isequipped with a much smaller display. Therefore, peers have to agree onterminal capabilities when negotiating QoS. Such terminal capabilitiesare e.g. the size of the display, the number of colors available, thegeneral capabilities of rendering a given media type (e.g. if an audiomixer is available), and all kind of information for services that arenecessary for rendering a given media type.

To this extent, the Composite Capabilities/Preference Profiles [CC/PP]framework language (a combination of XML and RDF) and request/responseHTTP protocol offer a handy way for specifying terminal capabilities inan RDF format.

Network Resource Reservation Support

It is useless to try to demand network resources end-to-end using asignaling protocol like RSVP, if it can be determined a priori that oneor multiple remote peers are not RSVP aware. To this extent, a possiblesolution is to ask peers to check if a given signaling protocol isavailable at their access networks.

A less complex solution is that, during pre-negotiation, peers indicateto each other along which direction they are going to reserve networkresources. Allowable values are N/A, sender, receiver, sender/receiver,where the concepts of sender and receiver relate to the resourcereservation protocol used (if any). This information can in fact modelthe type of reservation protocol used, if any.

For instance, in a simple Video-on-Demand service scenario with RSVPavailable end-to-end, the client acts as a receiver, whereas the serveracts as a sender, since RSVP is receiver-initiated. In case of YESSIR(see [Pan99]), which is sender-initiated, the configuration—from aresource reservation protocol perspective—would be the opposite.

Similarly, in a videoconference scenario with end-to-end RSVP support,each peer would be typically configured as a sender/receiver.

If no direction is specified for a given peer, the corresponding peerscan immediately deduce that no network resource reservation mechanism isavailable on the given peer. The key point is that this model allows tocope also with cases where no QoS signaling protocol (like RSVP) isavailable end-to-end. In such a case, each peer would be simply modeledas a sender (or nothing at all). This invention integrates the lattersolution.

QoS Contract

QoS Contracts are hereby defined as sets of high-level QoScharacteristics, each expressed in terms of any of the following:

-   -   Operating target: the desired value    -   Operating region: a desired interval of values    -   Thresholds/limits: a set of values marking different states with        respect to the QoS adaptation business logic.

Any of the aforementioned QoS specifications can eventually befurthermore expressed in a statistical format, by e.g. statingpercentiles, mean value, variance, etc.

Adaptation Path

Peers can agree not only on a given QoS Contract, but also onalternative ones, which can be advantageously used whenever the networkand/or terminal resources availability change over time. In such a way,each peer will exactly know which alternative QoS Contract (and underwhich circumstances) shall be enforced in order to cope with a criticalQoS Change or with any QoS Violation with respect to the currentlyenforced QoS Contract.

During the step (1) of the above-identified steps (1) to (4) onlystream-related Adaptation Paths are pre-negotiated. Higher level ones(associated with QoS Correlation issues) shall be negotiated at higherlevel at step (2) of the above-identified steps (1) to (4).

Together with the best-case QoS Contract, the Adaptation Path forms anextended QoS Contract. This extended QoS Contract is negotiated andindexed. If a peer later decides to switch to a lower level of QoS, onlythe index identifying the QoS Contract as a state of a hierarchicalFinite State Machine is needed to tell the other peer to adapt to thatQoS State.

Types of E2ENP

The actors participating to communication sessions can act roles like:

-   -   Sender/Receiver: the Sender can be solely transmitting data to a        Receiver, whereas a Receiver receives data from a Sender. A        two-way conference can be modeled with a Sender/Receiver and        Receiver/Sender pair.    -   Initiator/Responder: the initiator invites responders to        participate to a communication session. The responder waits for        a request from the initiator.

Possible combinations are:

-   -   Initiator and Sender→Responder and Receiver    -   Initiator and Receiver→Responder and Sender

In the case where the Initiator is the Sender, the Sender of a mediastream initiates the session (the Sender invites and the Receiver waitsto be contacted). In the case where the Initiator is the Receiver, theReceiver of a media stream initiates the session (the Receiver invitesand the Sender waits to be contacted). An Initiator/Sender can talk witha Responder/Receiver. An Initiator/Receiver can talk with aResponder/Sender.

In the following the key scenarios affecting the type of E2ENPconsidered in this invention are listed.

-   -   Peer-to-Peer negotiation: Can be actually extended also to        multiple peers participating to e.g. a videoconference, but        where each peering is regulated by a direct peer-to-peer QoS        negotiation mechanism.    -   1×N Multicast negotiation: In this case, a sender and multiple        receivers need to negotiate one or multiple QoS characteristics        at a time. This can be done either on a connection-wide basis        (i.e. determining a least common denominator good for all peers)        or on a receiver-selected basis (peer-to-peer negotiation for        some QoS characteristics among certain peers—see previous        point). In case of a best effort level of agreement, receivers        are still able to communicate even if they cannot agree on a        least common QoS level, otherwise (e.g. in case of a guaranteed        QoS policy), those receivers must be dropped or other mechanisms        have to be used. As an example, media filters or transcoders can        be used in between a Sender and a Receiver to tailor media        streams in such a way, that Receivers' capabilities and QoS        states can be matched with Senders' QoS states. The media        transcoder can be used to translate from one representation to        another (e.g. switching the codec of the media stream from PCM        to GSM audio). Media filters can be used for finer granular        adaptation of media qualities (e.g. adapting to frame size,        frame rate or desired quality without forcing the sender to        adapt).    -   The connection-wide E2ENP can be effectively used for enforcing        QoS Correlation among multiple flows from various sources. The        1×N Multicast negotiation may be used for determining the common        QoS not only for the case of a unique Sender transmitting to        multiple Receivers, but also for the case of an unique Receiver        correlating the QoS of incoming flows from many Senders.

ITU-T Recommendation X.641 (December /1997) ISO/IEC 13236:1998,Information technology—Quality of Service: Framework, describing theaforementioned types of negotiations, deals with 3-party negotiation,involving an Initiator, a Provider, and one or multiple Responders. Inthis invention we limit our scope however to the relationship between anInitiator and one or multiple Responders at application level. Thechoice of not directly involving explicitly Providers is due to the factthat Providers are only concerned with traffic and admission control.Therefore, only traffic contracts need to be negotiated between peersand the Network Providers. this can only be done during sessionestablishment, as it does not make sense to reserve resources(especially network ones) for a given period of time in advance, i.e.without using them.

Peers can therefore perform the hereby-described E2ENP pre-negotiationphase by agreeing upon nominal QoS levels. During actual connectionestablishment, peers then use the pre-negotiated contracts. These aretested locally, and resources are reserved. The result of this stage isthen offered to the peers, and any excess resources are eventuallyreleased. Finally, at this level, network resources are reserved, whichagain may lead to release of excess resources. By monitoring the actualconnection quality, peers can then decide at any time whether the givenpre-negotiated QoS contracts are satisfied. And if not, whether and howto select an alternate QoS level, based on the pre-negotiated adaptationpath. Provisions to avoid frequent requests to change network resourcereservations are put in place by properly introducing tolerances andhysteresis in the QoS contracts and adaptation paths, as described inthe European Patent Application 00 126 975.2 [Manda00b].

In any case, during the pre-negotiation phase, each peer assesses thebids received from other peers against pre-configured User-Profiles,which contain pre-computed Adaptation Paths. The assessment can beachieved by comparing each parameter of is each QoS Contract provided inthe bid, with the corresponding information contained in the UserProfiles. The decision whether to accept a bid or formulate acounteroffer is made with respect to all the QoS Dimensions of the bidQoS Contracts. Should the bid propose QoS Contracts with different QoSDimensions, only the subset of mutually understood QoS Dimensions wouldbe taken into account.

The initiator associates each QoS Contract of its bid with anidentifier. Responders will notify the acceptance of a given QoSContract (or the proposal for an update thereof, as a counteroffer), byspecifying the corresponding identifier. Should a Responder feature abetter QoS Contract resolution (like in the case of a given bid QoSContract, in correspondence to which multiple QoS Contracts with “finer”granularity, are available at one of the Responders), said Responder mayreturn a counteroffer with new QoS Contracts. It is up to the Initiatorto collect these counteroffers, choose the most meaningful ones,associate them with new identifiers, and propose them to all peers in anew negotiation round.

Mapping between Type of Service and Type of E2ENP

The following paragraphs describe how E2ENP can be deployed with respectto the various types of services mentioned above.

Conversational Services

This is the case of services like real-time videoconference or CSCW. Thekey characteristic of these services is that the type of contentexchanged among peers is most likely based on user interactions with thesystems and with other peers. For instance, most of video flows willcarry live images (e.g. taken from a webcam).

Another important aspect is that peers need to agree beforehand on agiven agenda, which includes the start-time and duration of the service.This is typically accomplished by using SAP and SIP, in the Internetworld. The manner how this type of services is actually deployed, ishighly implementation dependent. Taking a videoconference as an example,special bridges are used in the telecom world so as to offer a startopology solution. Any interested party can thus join a videoconferenceby simply contacting the bridge. In the Internet world on the otherhand, it is common the use of multicast technology: any interested partywould simply have to “tune” on the proper multicast group. From an E2ENPperspective, however, we hereby assume that peers shall use unicastsignaling for negotiating a common level of QoS.

Conversational services can be modeled as cases of one Initiator andmultiple Responders, where all peers can act as Sender and/or Receiver.Furthermore, from a resource reservation protocol, each party can useeither a sender-driven or a receiver-driven network resource reservationprotocol (if any). Thus, an Initiator requests some resources from thenetwork and from a Responder (since the Responder has to provide theresources for decoding and displaying). In addition, the Initiatormanages its own resources locally.

Distribution Services

Distribution services can involve a content provider and (i) known or(ii) unknown content consumers.

In the former case, SIP or RTSP extension might be used similarly to thecase of information retrieval services (see below), in the latter caseSAP extensions are hereby taken into account. The sender shall simplyannounce the list of multicast groups, which the receivers may choosefrom for “tuning” on the current content distribution.

Except for the case of known content consumers, only a reduced versionof E2ENP can be mapped to SAP. The Adaptation Path can be simply modeledas having the content provider streaming the same content with differentQoS levels to different multicast groups (one per QoS level). Contentconsumers shall therefore simply monitor the level of QoS available attheir terminal/access network, and eventually tune on another multicastgroup if any QoS Change or QoS Violation occurs in the meanwhile.Alternatively, the full-blown E2ENP could be used in conjunction withmedia filter/transcoder in the network, tailoring the media streams tomatch the different QoS levels. To this extent, the content consumersshould simply instruct the media transcoders/filters to switch to a newQoS level. It is up to the Content Provider and to the Receivers toensure that the streams distributed on different multicast groups aremutually synchronized.

For optimization purposes, the sender may however collect statisticsconcerning the number of receivers per multicast group (channel), so asto avoid flooding the network with packets of unused channels. Thisrequires signaling between the Content Provider and each Receiver. Thisinvention however does not address such cases.

Information Retrieval Services

This type of services is typically managed by using RTSP protocol. Thisinvention proposes enhancements of such protocol for achieving E2ENPgoals similarly to what described for the case of conversationalservices. More specifically, one has to note that the RTSP process isinitiated by the receivers by requesting the sender to describe a givenmedia (e.g. a video stream) through the RTSP DESCRIBE method. Throughthis method, a receiver gets informed about the various levels of QoS,which the sender may enforce for playing out the given media. Thereceiver has thus the possibility to choose which level to enforce atany given time. This is a form of pre-negotiation process paired with afast negotiation cycle, similar to the concept presented by thisinvention. The key idea is to enhance the TRSP DESCRIBE method toinclude all the information about capabilities described above, and tointegrate RTSP with the hereby-presented distributed resource managementmechanism.

QoS Correlation features can also be achieved by using SMIL [SMIL98].This invention does not address the integration of such technology.

End-to-End Re-Negotiation Protocol (E2ERP)

The E2ERP is the subset of the E2ENP focusing on re-negotiation issues.This protocol is activated whenever:

-   1. QoS Violations have been detected at functional entity involved    in the transmission process; or-   2. the current monitored level of QoS is no longer compatible with    the currently enforced QoS Contract; or-   3. a notification of newly discovered additional resources has been    received by some specific functionality; or-   4. one of the peers decided to enforce a different QoS Contract    (e.g. the user requested a higher frame-rate).

Cases 2 and 3 are mostly related to optimization processes, which allowapplications to upgrade/degrade their QoS requirements in a smoothmanner, whereas case 1 deals with dramatic shortages of resources, whichbadly affect applications, if not properly and promptly treated.

To this extent, case 1 implies re-negotiating local and eventuallynetwork resources without the need to try to maintain the previouslyreserved resources, whereas this is not possible in cases 2 and 3. Thismeans that whenever dealing with optimization issues, one has to takecare of:

-   -   either allocating resources in excess of the difference between        the previously allocated resources and the new amount (in the        case of QoS upgrade);    -   or allocating the new resources from the scratch, while keeping        the old ones until the re-negotiation has been successfully        completed (pessimistic approach). This can also be modeled as        trying to allocate more resources while still keeping the old        one. If this re-allocation fails, the originally reserved        resources are still reserved, otherwise the newly allocated        resources are available;    -   releasing some resources to switch to a lower QoS state (in the        case of QoS downgrade).

The first option seems to be quite attractive, but some resources mightnot be easily shared among different QoS States. For example, thescheduling information about the thread of a given codec might betotally useless, if the codec is going to be preempted and substitutedwith another one, which runs on a totally different thread.

Therefore the second approach, though pessimistic, seems to be the mostrealistic one. In the following we will therefore refer explicitly tothe allocation of new resources and later release of old ones. Inaddition, releasing resources is always possible, because the systemload is decreased.

Whenever a re-negotiation is triggered, each peer determines from thepre-negotiated Adaptation Path the best alternative QoS Contract thatshould be enforced in order to cope with the detected QoSChange/Violation. The identity of the new QoS Contract is exchangedamong peers, in order to come to an agreement on which QoS Contractshould be actually enforced. Having previously negotiated the AdaptationPath allows the peers to more efficiently perform such re-negotiating,by simply exchanging identifiers. Should all peers detect a violationsimultaneously, some form of contention resolution algorithm would needto be enforced. To this extent different well-known techniques areavailable: assigning master/slave roles (typically assigning to theInitiator the master role), or re-attempting after a random time haselapsed. This invention leverages the master/slave mechanism.

In summary, the following types of re-negotiation are identified:

-   a) re-negotiation-triggered by a QoS violation detection, where new    resources are allocated from the scratch and the old resources are    automatically released (for QoS downgrade cases, this might be not    an issue);-   b) re-negotiation for optimizing the system, where new resources are    allocated from scratch and the old resources are released later, at    E2ERP successful completion; and-   c) re-negotiation for optimizing the system, where new resources are    allocated from scratch and the old resources are automatically    released.    A framework for Dynamic End-to-end QoS Negotiation and Control

This invention presents a framework for achieving dynamic end-to-end QoSnegotiation and control coordination, for distributed multimediaapplications. The framework builds upon dynamic capability negotiationand specification of Adaptation Paths and QoS Contracts, based on userpreferences. These principles will now be explained with reference tothe figures.

The Initiator 101 finds the protocol version using a Protocol DiscoveryUnit 104, 213 via interface 214, which queries a Directory Server 102,215. This is coordinated with the local Coordination Unit 204. TheSession Protocol Unit 205 is used for all kind of negotiations andcommunication. In particular it negotiates with the Peer 216 (which isthe Responder 103) a common set of capabilities and configurations andthus builds a vocabulary in the Pre-Negotiation phase 106. It alsonegotiates QoS Contracts and Adaptation Paths that are used for theMulti-Stream QoS synchronization and QoS correlation phase 107. LocalResource management is carried out by the Local Resource ManagementUnits (responsible for CPU, buffer and other local resources) 206 viainterface 211. Local Resource management is coordinated by theCoordination Unit 204 with network resource reservation, which iscarried out by the Network Resource Reservation Unit 207, via interface212.

Any end-to-end interaction during communication session establishment,i.e. during the Fast Negotiation phase 108, is carried out by followingthe Economy Principle. This principle states that first local, then peerand finally network resources are reserved. If during the communicationsession a re-negotiation is necessary due to a change in resourceavailability or to any other reasons influencing resource availability,the Re-Negotiation phase 109 is carried out by following the EconomyPrinciple as well.

Finally, resource reservations release is done during the Release phase110. The local Protocol Stack 208 interfaces with the Session ProtocolUnit 205 via the interface 217 and implements the session protocol. Inaddition, the Protocol Stack 208 implements the network resourcereservation protocol which is controlled via interface 210.

The QoS-Aware Application Unit 202 uses a FSM Engine Unit 203 to controlthe states of all the streams that the QoS-Aware Application Unit 202handles. The Coordination Unit 204 together with FSM Engine Unit viainterface 209 provides the QoS-Aware Application Units 202 with thepossibility to establish, use, control and release network connectionswith other applications, using QoS guarantees. Establishing suchconnections involves negotiation of capabilities, configurations, QoSContracts and Adaptation Paths (managed by the FSM Engine Unit 203), andintegration of local, peer, and network resources according to theEconomy Principle. Control involves re-negotiation and integration oflocal, peer, and network resources and may involve a state transitionmanaged by the FSM Engine Unit 203, in cooperation with the CoordinationUnit 204. Release involves network and local, as well as peer resourcetear-down and state management (by the FSM Engine Unit 203), incooperation with the Coordination Unit 204.

This invention also presents the concept of the E2ENP Broker 105, anoptional third-party entity, which can be used for relieving peers 101,103 from performing the time- and resource-consuming Pre-Negotiationphase 106 (and eventually also the Multi-Stream QoS Synchronization andQoS Correlation 107). Each peer 101, 103, after discovering from aDirectory Server 102 which E2ENP Broker 105 to contact, can upload itsAdaptation Path, Capabilities, and the list of peers to contact on saidE2ENP Broker 105. The E2ENP Broker 105 may be implemented e.g. as anenhanced version of audio-/videoconference bridges, or by combining anenhanced SIP proxy server (or SIP redirect server)—dealing with thenegotiation process—and an enhanced version of a SIP registrar—storingthe Adaptation Path and Capabilities and all the intermediate negotiatedversions thereof.

The E2ENP Broker 105 thus collects such information from each peer 101,103, and performs the negotiation process locally in its memory space.During this process, the E2ENP Broker 105 may eventually contact some ofthe indicated responders 103 on behalf of the initiator(s) 101, shouldsuch responders 103 do not have yet stored their information on saidE2ENP Broker 105. Once the negotiation has been carried outsuccessfully, the E2ENP Broker 105 will notify the peers 101, 103 aboutthe availability of the results, and each peer 101, 103 can thendownload said results. The E2ENP Broker 105 can also allow multiplenegotiation iterations (otherwise too expensive to be carried out amongpeers themselves). Responders 103 might disagree with a bid from theInitiator 101, and therefore provide counteroffers. The analysis ofcounteroffers involves multiple round-trip message exchanges amongpeers, and therefore is too time consuming. The E2ENP Broker 105 canmediate all this process.

EXAMPLE

The example in this section is illustrating the concepts of ourinvention. It is based on a real-time multimedia conferencing scenariousing IP-based networks. Media transport is carried out by usingRTP/RTCP.

For simplicity reasons, we are showing the invention's principle using apseudo-protocol (QUERY, RESPONSE, COMMIT, START, START_ACK, RESERVE,CONFIRM) which can be mapped onto existing SIP/SDP plus extensions to bestandardized. Also, this example is restricted to two peers A and B. Letus assume that the Initiator 101 wants to receive a media stream (let usassume video) from the Responder 103.

The E2ENP process starts with the Pre-Negotiation phase 106 assumingthat the Protocol Discovery 104 has successfully been completed earlier,figuring out a common E2ENP version that both parties understand.

In the Pre-Negotiation phase 106, the QoS-Aware Application Unit 202requests-a negotiation using the local FSM Engine Unit 203, providing abid. This bid contains a list of capabilities and configurations as wellas QoS Contracts and the Adaptation Path(s), which are all indexed. Thedescription of the bid can be realized using SDP extensions (e.g. as anew feature of the currently investigated SDPng [DPNG00]).

The Coordination Unit 204 forwards the bid via the session protocol unit205 to the peer using the QUERY message, which can be based on SIPextensions. The peer's Session Protocol Unit 205 receives the QUERY andasks the Peer Coordinator 204 to evaluate the bid using the Peer FSM203. The Peer FSM matches the bid with its own adaptation path andcapabilities and generates a common subset, which is forwarded in theResponse massage via the Peer Coordinator 204 to the Peer SessionProtocol Unit 205, which generates a RESPONSE message which can be basedon SIP extensions. The initiator's Session Protocol Unit 205 uponreception of the new bid forwards an evaluate request to the local FSM203 via the Coordinator 204. Finally, the new bid is again checked forcompatibility (it might be empty indicating that the peers areincompatible) and the Initiator's (101) FSM Engine 203 collects all thenew bids from all responders and generates a common subset, which issent to all peers. This is done by first sending an enforce message to204, which forwards it to 205. The initiator's Session Protocol Unit 205generates the commit message, which can be based on SIP extensions. Thismessage carries the bid-match result (which can be described using SDPextensions). This is distributed to the session protocol unit 205 of theresponder 103, which enforces the result via the Coordination Unit 204to the FSM 203. At this point, all communication partners have agreed ona common set of codecs, capabilities, adaptation paths and QoS statesand are able to decode later on the references (denoted as StateIDs).

The assumption which shall be made now is that the initiator 101 wantsto initiate the actual session, which contains one video stream. Thestates and adaptation paths have been negotiated during thepre-negotiation phase 106 just described. The QoS-Aware Application Unit202 of the Initiator 101 requests a start of the stream given a StreamIDand its desired StateID from the local Coordinator Unit 204, which firstperforms local admission tests and reserves the resources locallyrequesting the resources from the Local Resource Management Units 206.If the desired state cannot be fulfilled due to local resource problems,the Coordinator Unit 204 would request the FSM Engine 203 to try thenext QoS State having lower resource requirements according to thenegotiated Adaptation Path. After local successful resource reservation,the start request is propagated to the local Session Protocol Unit 205,which sends a START message to the peer's Session Protocol Unit 205message, which can be based on SIP extensions. This message contains theenforced StateID and Stream ID (which both can be modeled by using SDPextensions. The start message is forwarded via the peers CoordinatorUnit 204 to the peers FSM Engine Unit 203, which checks for a validStateID. If this is approved, the FSM Engine requests resources forhandling the media streams using the Coordination Unit 204 and theservices provided by the Local Resource Management Units 206, whichperform admission tests and reserve the resources at the peer.

Note that if the resources at the peer cannot be granted, the peerCoordination Unit 204 requests the FSM Engine Unit 203 to select a newStateID according to the pre-negotiated Adaptation Path. If everythingis approved, the peer Coordination Unit 204 requests the peer SessionProtocol Unit 205 to send an START_ACK to the Initiator's 101 SessionProtocol Unit 205 with the new StateID, which is forwarded via the localCoordination Unit 204 to the local FSM Engine Unit 203. The local FSMEngine Unit 203 checks for compatibility of the states and, if approvedtells the local Coordination Unit 204 to request reservation of networkresources (e.g. using RSVP) via the local Network Resource ReservationUnit 207. If the StateID were different from the originally selectedone, it would instruct the Local Resource Management Units 206 to updatethe local resource reservation according to the new StateID (which isnot shown in FIG. 2, because we assumed that everything was fine).Additionally, the Coordination Unit 204 may request the Session ProtocolUnit 205 to send a RESERVE message to the peer 103, should that peer beresponsible to manage network resource reservation within the accessnetworks they are attached to [Marsh00]. In such a case, upon receptionof the RESERVE message through its Session Protocol Unit 205, the peer103 would trigger its Coordination Unit 204 to request network resourcesvia the peer's Network Resource Reservation Unit 207 (e.g. using RSVP).Once resources have been reserved at the network level, the CoordinationUnit 204 at the peer 216 sends a confirmation back to the sender usingthe CONFIRM message, which is received by the local Session ProtocolUnit 205. This unit generates a confirmation to the local CoordinationUnit 204. Once the Initiator 101 has collected all the confirmationsfrom all the Responders 103, it releases the role of the Initiator 101and everything is setup properly.

For performing concurrency control, anybody starting a Pre-Negotiationphase 106, Multi-Stream QoS Synchronization and QoS Correlation phase107, Fast negotiation phase 108, or Re-Negotiation phase 109 can takethe role of Initiator 101. Therefore it will start sending messages toknown peers inviting them to participate in the given phase. If none ofthe other peers is acting as Initiator 101, Peer 216 realizes it isacting as an Initiator 101 and waits confirmation of resourcereservation (aggregate local and network) from Peers acting asResponders 103. Otherwise, if one of the Peers 216 is already acting asInitiator 101, said Initiator 101 will add the prospecting Peer 216 toits list of Responders 103, and the prospecting Peer 216 will wait toget invited by said Initiator 101.

Once the given phase is over, i.e. once the given Initiator 101 hascollected all the confirmations from all the Responders 103, the givenpeer acting as Initiator 101 “releases” the role of initiator. In thismode a token passing mechanism is used, where the acquisition of thetoken grants the role of Initiator 101 and the passing is done once aphase is completed. In this way also the case of collisions is coveredby using a master-slave approach, where the master is the Initiator 101and the slave is the Responder 103. The Initiator 101 commands theResponder 103 when to start network reservation. In any case, networkreservation would be carried out concurrently among peers (except forthe case of a QoS reservation signaling protocol—like RSVP—whereavailable end-to-end).

ASPECTS OF THE PRESENT INVENTION

Now the different concepts proposed by the present invention will beexplained:

-   -   Integration and extensions of different existing protocols and        technologies for defining the E2ENP (including E2ERP): More        specifically, this aspect encompasses the following phases:        Protocol Discovery 104, Pre-Negotiation 106, Multi-Stream QoS        Synchronization and QoS Correlation 107, Fast-Negotiation (with        Economy Principle) 108, Re-Negotiation (with Economy Principle)        109, Resource Reservation Release 110. All the six phases can be        concatenated, or be executed at different times. The        Multi-Stream QoS Synchronization and QoS Correlation 107 phase        is optional, insofar as it is required only if Peers 216        interact by sending/receiving multiple streams, which need to be        correlated and synchronized. The Protocol Discovery 104 and        Pre-Negotiation 106 phases can be executed a priori, and the        results can then be applied to multiple successive communication        sessions, whereby each communication session is initiated with a        specific Multi-Stream QoS Synchronization and QoS Correlation        phase 107. Should even the results of the latter phase be        applicable to multiple successive communication sessions, each        of said communication sessions can be initiated with a specific        Fast Negotiation phase 108. The protocol interacts with the        Local resource management units 206 during Pre-Negotiation 106,        Multi-Stream QoS Synchronization and QoS Correlation 107, Fast        Negotiation 108, Re-Negotiation 109 and Resource Release 110        phases.    -   Economy Principle: According to this aspect, the resource        admission control and resource reservation can be applied        following this order: On the E2ENP Initiator 101 first, then on        the E2ENP Responder(s) 103, and finally on the network using        207. In the first two steps, issues like real-time CPU        scheduling, memory management, and power management are        addressed. The same process applies in case of re-negotiations,        with the difference that previous allocated local resources need        to be updated (either released or re-allocated) either        immediately or later during the Resource Reservation Release 110        phase. For the Resource Reservation Release 110 phase, first        network resources, then peer, and finally local resources are    -   Concept of E2ERP: According tot his aspect, during runtime of a        so established multimedia session, at any time any component may        request an adaptation. This again requires coordination of        local, peer and network resource management, according to the        Economy Principle. When performing this adaptation, user-defined        or user-provided preferences can be used implicitly in order to        build the Adaptation Path.    -   Pre-negotiation of type of E2ENP: This can be done during        Protocol Discovery phase 104, either by forcing peers 101, 103        to query a Directory Server 102, or by having the latter        announcing such information. Instead of a directory server, a        SIP registry server can also be used.    -   A Pre-negotiation of capabilites: This can be done during        Pre-Negotiation phase 106, by using specific protocol, e.g.        SIP/SDP plus extensions.    -   A Pre-negotiation of complete codec list: This can be done        during Pre-Negotiation phase 106, by using specific protocol,        e.g. SIP/SDP plus extensions.    -   Pre-negotiation of Adaptation Paths at stream level: This can be        done during Pre-Negotiation phase 106, by using specific        protocol, e.g. SIP/SDP plus extensions.    -   Pre-negotiation of Adaptation Paths at stream aggregation level:        This can be done during Multi-Stream QoS Synchronization and QoS        Correlation phase 107, e.g. SIP/SDP plus extensions.    -   Indexing of pre-negotiated QoS Contracts and Capabilities for        speeding up the Fast Negotiation phase 108.    -   Indexing of pre-negotiated QoS Contracts and Capabilities for        speeding up the Re-Negotiation phase 109.    -   Modeling the data model, described in EPA 00 126 975.2, in an        extended version of either SDP or SDPnG. Handling        installation/de-installation of Capabilities even at runtime, by        exchanging asynchronous messages among peers for notifying such        events. This can be based on e.g. SIP/SDP plus extensions.    -   Support of a E2ENP Broker 105: this is an optional third-party        entity, which can be used for relieving peers 101, 103 from        performing time- and resource-consuming Pre-Negotiation phase        106 (and eventually also the Multi-Stream QoS Synchronization        and QoS Correlation 107). Each peer 101, 103, after discovering        from a Directory Server 102 which E2ENP Broker 105 to contact,        can upload its Adaptation Path, and the list of peers to contact        on said E2ENP Broker 105. The E2ENP Broker 105 may be        implemented e.g. as an enhanced version of        audio-/videoconference bridges, or by combining an enhanced SIP        proxy server (or SIP redirect server)—dealing with the        negotiation process—and an enhanced version of a SIP        registrar—storing the Adaptation Path and Capabilities and all        the intermediate negotiated versions thereof. The E2ENP Broker        105 thus collects such information from each peers 101, 103, and        performs the negotiation process locally in its memory space.        During this process, the E2ENP Broker 105 may eventually contact        some of the indicated responders 103 on behalf of the        initiator(s) 101, should such responders 103 do not have stored        their information on said E2ENP Broker 105. Once the negotiation        has been carried out successfully, the E2ENP Broker 105 will        notify the peers 101, 103 about the availability of the results,        and each peer can then download the results.    -   Use of E2ENP Broker 105 to allow multiple negotiation        iterations, which would otherwise be too expensive. Responders        103 may disagree with a bid from the Initiator 101, and        therefore provide-counteroffers. The analysis of counteroffers        involves multiple round-trip messaging among peers, and        therefore is too time consuming. The E2ENP Broker 105 can        mediate and speed up the given process.    -   Extension of SIP and RTSP for Pre-Negotiation 106 of Adaptation        Path (at stream level) and Capabilities, by piggybacking this        information in existing messages of SIP or RTSP.    -   Extension of SIP and RTSP for Pre-Negotiation of Adaptation Path        at stream aggregation level 107, by piggybacking this        information in existing SIP or RTSP messages (according to EPA        00 111 191.3).    -   Protocol procedures and messages for addressing Fast Negotiation        108 and Re-Negotiation 109 phases (by using the identifiers        associated with the pre-negotiated QoS Contracts of the        Adaptation Path).    -   Extension of SIP/SDP and RTSP, by defining messages for        addressing:        -   a. QoS Contract negotiation;        -   b. coordination of network resource reservation (if any)            among peers.    -   Extension of SIP and RTSP for Fast Negotiation 108, by using        indexed QoS Contracts and Capabilities.    -   Extension of SIP and RTSP for Re-Negotiation 109, by using        indexed QoS Contracts and Capabilities.    -   Coordination of network resource reservation mechanisms among        peers, according to the Economy Principle, by introducing        acknowledged synchronization procedures in SIP and RTSP during        Fast Negotiation 108 and Re-Negotiation 109 phases.    -   Extension of SAP for allowing Directory Servers 105 announcing        types of end-to-end QoS negotiation protocols used.    -   Extension of existing directory protocols or of SIP with respect        to the use of SIP Registrars, in order to store and/or retrieve        types of end-to-end QoS negotiation protocols used for enabling        the Protocol Discovery Phase 104.    -   Extension of SAP for announcing Adaptation Paths with respect to        Distribution Services, where AP are simply listing various        multicast groups, each targeting a specific QoS Contract.    -   All the E2ENP-related changes to SIP can theoretically be mapped        onto H.323 as well.    -   Management of resource reservation release 110    -   The negotiation and re-negotiation of capabilities—during,        respectively, the Fast-negotiation (with Ecoomy principle) phase        108 and the re-negotiation (with Economy principle) phase        109—include the signaling that the peers 216 need to exchange        with each other, in order to agree on the choice of audio and/or        video codec and the configuration thereof (bitrate etc.)

EMBODIMENT FOR THE IMPLEMENTATION OF THE PRESENT INVENTION

Now an embodiment for the implementation of the present invention willbe explained with reference to FIG. 2:

Each Peer 216 consists of a Computing Unit 201 containing a ProtocolStack 208 (e.g. IP and TCP/UDP) and a Coordination Unit 204. TheCoordination Unit 204 coordinates.

-   1. Protocol Discovery phase 104 through interface 214 to Protocol    Discovery Unit 213;-   2. Pre-Negotiation phase 106 and Multi-Stream QoS Synchronization    and Correlation 107 through interface 209 to FSM Engine Unit 203,    and through interface 210 to Session Protocol Unit 205;-   3. Fast Negotiation phase 108 through interface 211 to Local    Resource Management Units 206, through interface 210 to Session    Protocol Unit 205, and through interface 212 to Network Resource    Reservation Unit 207;-   4. Re-Negotiation phase 109 through interface 209 to FSM Engine Unit    203 (for evaluating alternative QoS Contracts by using the    Adaptation Path), through interface 211 to Local Resource Management    Units 206, through interface 210 to Session Protocol Unit 205, and    through interface 212 to Network Resource Reservation Unit 207;-   5. Release phase 110 through interface 211 to Local Resource    Management Units 206, through interface 210 to Session Protocol Unit    205, and through interface 212 to Network Resource Reservation Unit    207.

A FSM Engine Unit 203 can be part of a given QoS-Aware Application Unit202, or can be a separate entity (middleware), which multiple givenQoS-Aware Application Units 202 can use. The FSM Engine Unit 203 is asoftware process that, after having been configured with a givenAdaptation Path, is capable of implementing the FSM associated with thegiven Adaptation Path, on behalf of the QoS-Aware Application Unit 202,in cooperation with the Coordination Unit 204.

A Protocol Discovery Unit 213 allows the Coordination Unit 204contacting a Directory Server 215 (which might also be implemented as anextension of a SIP Registrar), in order to accomplish the ProtocolDiscovery phase 104. The Protocol Discovery Unit 213 extends the use ofexisting specific protocols (e.g. LDAP, SAP, SIP), by advertising andquerying information about E2ENP, on behalf of the Coordination Unit204.

A Session Protocol Unit 205 allows the Coordination Unit 204 carryingout the phases 106-110 with Peers 216. The Session Protocol Unit 205extends current protocols like SIP/SDP or RTSP (or even H.323/H.245), byusing SDP or SDPnG enhancements for describing bids/counteroffers forAdaptation Path and Capabilities, and by introducing SIP or RTSP (oreven H.323/H.245) specific messages for implementing the signalingrequired for the phases 106 to 110, especially for the Pre-Negotiationphase 106 and the Multi-Stream QoS Synchronization and Correlation phase107 (including the resource coordination according to the EconomyPrinciple).

Interfaces 211 to the Local Resource Management Units 206 are identifiedwith respect to the following classes of resources:

-   -   CPU resources    -   Memory (primary and secondary) resources    -   Battery power    -   External auxiliary resources

For each class of resources, a specific Local Resource Management Units206 is made available, e.g. as an extension of the Operating Systemloaded onto the Computing Device 201.

The interfaces 211 allow the Coordination Unit 204 to carry out thefollowing tasks:

-   -   Monitor resource usage, according to various criteria (e.g.        ordered per stream or per process)    -   Register (and de-register) for notifications to be forwarded to        the Coordination Unit 204 once and/or anytime a given condition        is met    -   Reserve (and release) a given amount of resources (by e.g.        setting a CPU deadline or pinning a memory area to a given        process)    -   Update a given reservation (for the re-negotiate case)    -   Associate (and de-associate) a given entity, e.g. a process, to        a class of local resources usage (this is typically done for        those resources that cannot be directly manipulate without OS        supervision). The class is a kind of priority, which the OS can        use for optimizing resource usage according to the user's        policies.

Each Peer 216 can take the role of Initiator 101, provided that allother peers are not acting as Initiator 101 in the meanwhile. The roleis maintained for the whole duration of any of the following phases:

-   -   Pre-Negotiation 106,    -   Multi-Stream QoS Synchronization and QoS Correlation 107,    -   Fast-Negotiation (with Economy Principle) 108, and    -   Re-Negotiation (with Economy Principle) 109.

During the Resource Release phase 110, some of the peers not involved inthe session tear down process—and thus willing to continue the sessionwith any remaining peer—may also take the role of the Initiator 101 in aconcurrent phase 107, 108 or 109.

Should a Peer 216 (thereinafter the peer A) try to act as an Initiator101 and invite other Peers 216 to act as Responders 103 for one of theaforementioned phases, while said phase is ongoing (i.e. while one ofthe other Peers 216 is already acting as an Initiator 101), the peer Awould be notified that it needs to wait an invitation from the currentlyactive Initiator 101, and the latter would have to add peer A to itslist of Responders 103.

The conclusion of any of the aforementioned phases coincides with thecollection at Initiator 101 side of all the confirmations ofsuccessfully reserved network resources from all the Responders 103. TheInitiator 101 has to validate each confirmation against to all theothers already received, in order to eventually rebalance the use ofresources among peers, by taking corrective actions.

Once the conclusion has been reached, the Peer 216 acting as Initiatorenters in a neutral state, thus releasing the role of Initiator 103 toany prospective Peer 216. The role of Initiator 103 can be thus modeledas a logical token (the Initiator Role Token IRT), which can beexchanged among peers once any of the aforementioned phases is over.

Other Peers 216 can join an already active communication session amongmultiple Peers 216. Is it really necessary and/or feasible in thesecases to re-negotiate everything? One key point of this invention is thede-coupling of the Pre-Negotiation 106 phase from the actualMulti-Stream QoS Synchronization and QoS Correlation 107,Fast-Negotiation (with Economy Principle) 108, and Re-Negotiation (withEconomy Principle) 109 phases. As long as Peers 216 agree beforehand, atsession advertisement time, on what level of QoS to use, they can thenjoin (or leave) an active session anytime, by using the fast indexednegotiation process described in this invention. The key problem,however, is how to deal with Peers 216 joining the conference at a latertime, without having pre-negotiated anything. In this case, severaloptions are available:

-   1. To re-negotiate everything from scratch among all Peers 216—this    is an unfeasible solution, not only because of the service    disruption with high latency, but also because this solution would    enforce dependencies among the connections belonging to e.g. a    videoconference session, which is unrealistic in most of the cases.-   2. To force the new Peers 216 to accept the already pre-negotiated    information: If a new Peer 216 does not accept this, he/she can not    join the given session.-   3. To enforce media adaptation through some filters embedded into    the network, to meet the other preferences of the new Peer 216. More    details on this concept can be found in [Pasqu92], [Pasqu93],    [Yeado93], [Kassl99a]) and [Kassl99b].

As a conclusion, this invention assumes that either solution 2 or 3 isenforced by any application enabled to handle these issues (e.g. aconference control tool), and using the services of this invention thismeans that the solutions 2 and 3 are outside the scope of thisinvention.

Should the E2ENP Broker 105 be used and/or the negotiated vocabulary bestored in the Directory Server 102 or SIP registrar, the newparticipants can more advantageously get directly the pre-negotiatedinformation from such entities, and thus trigger only theFast-Negotiation (with Economy Principle) 108, Re-Negotiation (withEconomy Principle) 109.

The temporary asymmetric Initiator 101/Responder 103 scheme isequivalent to a master/slave configuration, where the Initiator 101 actsas a master, and each Responder 103 as a slave.

During one of the Pre-Negotiation 106, Multi-Stream QoS Synchronizationand QoS Correlation 107, Fast-Negotiation (with Economy Principle) 108,and Re-Negotiation (with Economy Principle) 109 phases it is thereforepossible to resolve possible conflicts arising from Peers 216independently initiating a phase.

During the normal conversation phase (i.e. after the Fast-Negotiation108 phase), multiple Peers 216 can detect any QoS Change/QoS Violationsimultaneously and this can lead to collisions in requesting to startcorrespondingly the Re-Negotiation 109 phase. To this extent, the lastPeer 216 that took the role of Initiator 101 shall take again such arole and arbitrate the requests among peers. This means that each Peer216 shall retain the identity of the Peer 216 that last acted asInitiator 101.

As a worst case, should however such a Peer 216 have left in themeanwhile the communication session, the collision would be inevitable.In such a case, each Peer 216 trying to “seize” the IRT, upon receptionof similar request from other Peers 216 shall block for a random time,and retry when such a time has elapsed. If during this time, anotherPeer 216 has sent a request to seize the IRT, the Peer 216 suspended onsaid timer would be notified accordingly, and consequently it shallabort both the timer and its request to seize the IRT, and consequentlytake the role of Responder 103.

In FIG. 5 an example for the implementation of the E2ENP Broker conceptis shown, in which the Session Initiation Protocol (SIP) is applied asalready mentioned in other parts of this application. SIP Proxy Serversand/or SIP Redirect Servers 501 are associated with SIP Registrars 503.This invention proposes to augment the location information 505 storedin the databases 504 used by the SIP Registrars 503 by including the QoSSpecification and capabilities information 506 described in thisinvention. As well as the negotiated QoS information 507 (both thetemporary results being drawn during negotiation/re-negotiation phases,and the ones finally agreed upon). Whenever registering with the SIPRegistrar 503, users can thus upload the information 506 and 507. TheE2ENP Broker 502 is a unit which communicates with the SIP Proxy Serversand/or SIP Redirect Servers 501 over a specific interface 508 by meansof well known mechanisms (e.g. a JAIN SIP Servlet API): Whenever a E2ENPprotocol implemented as a SIP extension is used, the E2ENP Broker 502performs the negotiation and re-negotiation phases described in thisinvention in its memory space on behalf of all the peers by using theaforementioned information stored in the Registrar's database. The E2ENPBroker 502 communicates with the SIP Registrar 503 over the interface509.

THE MAIN ADVANTAGEOUS DIFFERENCES BETWEEN THE INVENTION AND THE STATE OFTHE ART

Technologies Comment [Bor00] The hereby-proposed mechanism for managingconference state during sessions and finding appropriate configurations,by negotiating for configurations and changing them, are based on thisstate of the art. This invention integrates this concept with local,peer and network resource reservation according to the economyprinciple. The invention introduces additional Adaptation Paths and QoSContracts and references these using handles. Also, this inventionincludes terminal capabilities and network resource reservation supportinto the negotiation mechanisms. [Burn00], The hereby-proposed mechanismto integrate local and [Kass100] and remote as well as network resourcemanagement in a [Kass100a] framework is based on this state of the art.In addition, this invention defines the protocol that is necessary fornegotiation of capabilities, preferences and QoS states and alsoidentifies different versions of the protocol using a directory serverapproach or SIP registrar. [Nahr95] The hereby-proposed economyprinciple is based on this state of the art. This invention integratesthis concept with existing Internet protocols, includes Adaptation Pathsand Indexes and considers other resources like battery power or wirelesssub- network availability. This invention extends the work by firstpre-negotiating several candidate QoS Contracts and configurations thusbuilding a common vocabulary. [RFC2533] The hereby-proposed mechanism torepresent media handling together with capabilities, express mediafeature sets and match feature sets [CONNEG00], (compare the bid withthe local capabilities) is based on this [CONNEG00a] or state of theart. This invention integrates this concept with [SDPCN01] existingInternet protocols, includes Adaptation Paths and integrates local aswell as network and peer resource reservation. This invention extendsthe work by first pre- negotiating several candidate QoS Contracts andconfigurations thus building a common vocabulary. [RFC2703] Thehereby-proposed mechanism for protocol independent negotiation of QoScontracts and capabilities is based on this state of the art. Thisinvention however negotiates QoS contracts, adaptation paths andcapabilities instead of content as in [RFC2703]. This invention alsointegrates this concept with existing Internet protocols, includesAdaptation Paths and integrates local as well as network resourcereservation. This invention extends the work by first pre-negotiatingseveral candidate QoS Contracts and configurations thus building acommon vocabulary [SDPNG00] The mechanism proposed in this invention toperform together with endpoint capability negotiation and include userpreferences [SDPCN00], and network resource reservation is based on thisstate of the [Ott99], art. [SDPCN00] adds SDP extensions for capability[Beser00] negotiation, only. [Ott99] only provides a notation forconfiguration description. [Beser00] only provides SDP extensionsnecessary for endpoints to negotiate codecs. This invention adds thenegotiation protocol to determine a common set of configuration andintegrates local, peer and network resource reservation by observing theEconomy Principle. In addition, the invention integrates the process ofreferencing configurations by handles and builds upon QoS contracts.Also, this invention not only negotiates codecs, but also terminalcapabilities and network resource reservation mechanisms. [SIPRES01],The hereby-proposed invention is based on the integration of togetherwith SIP/SDP with network resource reservation as a precondition.[Cama00], This invention extends the work by first pre-negotiating[Cama00a] several candidate QoS Contracts and configurations thusbuilding a common vocabulary. Also, only references to theconfiguration/QoS contract are referred to later on. In addition, thisinvention integrates local and peer resource management with the conceptprovided by this state of the art by observing the Economy Principle.This state of the art only covers one-to-one communication with resourcereservation and does not cover joining operations performed duringongoing sessions with negotiations. [Levin01] The hereby-proposedinvention fulfills many of the requirements stated in this state of theart. In addition, this invention provides the protocols and themechanisms to integrate local, remote and network resource managementinto a coherent framework observing the Economy Principle. [Levin01]only provides requirements and no protocol or mechanisms to enforce therequirements.

1. A method for exchanging media streams between peers of a network, themethod comprising: pre-negotiating a common set of a plurality ofindexed Quality-of-Service (QoS) contracts and terminal capabilitiesbetween peers, before streams are exchanged, wherein the pre-negotiatingincludes forwarding, from an initiator peer, indexed QoS contracts andterminal capabilities to at least one responder peer; generating, at theat least one responder peer, a common subset of said received indexedQoS contracts and its own QoS contracts and terminal capabilities; andforwarding, from the at least one responder peer, the common subset tothe initiator peer; and negotiating, at the peers, QoS contracts on aper stream basis using the prenegotiated QoS contracts and terminalcapabilities, at the time of stream establishment, wherein in case of arenegotiation during streaming, one of the peers sends the index of oneof the indexed pre-negotiated QoS contracts and terminal capabilities tothe other peers.
 2. The method of claim 1, further comprising:negotiating Quality-of-Service correlation and synchronization aspectsamong multiple streams among multiple peers.
 3. The method of claim 1,wherein in case of a renegotiation during streaming the peers refer to apre-negotiated state, and said state refers to a given pre-negotiatedquality-of-service contract and a given pre-negotiated set of terminalcapabilities.
 4. The method of claim 1, wherein peers negotiate duringpre-negotiation at the finest level of resolution, sets ofquality-of-service contracts on a per-stream basis and/or on aper-stream-association basis, and said stream associations are bundlesof streams from one sender peer to a receiver peer.
 5. The method ofclaim 1, wherein peers are informed about changes in capabilityconfiguration.
 6. The method of claim 1, wherein pre-negotiatedalternative quality and configuration information of a given type ofstream may already be available at a server for clients to choose from.7. The method of claim 1, further comprising: Protocol Discovery,Pre-Negotiation, optional Multi-Stream QoS Synchronization and QoSCorrelation, Fast-Negotiation, renegotiation and Resource ReservationRelease.
 8. The method of claim 7, wherein all phases are concatenatedand can be executed either continuously or at different times.
 9. Themethod of claim 8, wherein the Multi-Stream QoS Synchronization and QoSCorrelation phase is optional and required only if an Initiatorcommunicate with multiple peers by using multiple streams, which need tobe correlated and synchronized, based on user policies to be enforced atthe Initiator side only.
 10. The method of claim 9, wherein the ProtocolDiscovery and Pre-Negotiation phases are executed a priori, and theresults can then be applied to multiple successive communicationsessions, whereby each communication session is initiated with aspecific optional Multi-Stream QoS Synchronization and QoS Correlationphase.
 11. The method of claim 10, wherein, if the results of theMulti-Stream QoS Synchronization and QoS-Correlation phase areapplicable to multiple successive communication sessions, each of saidcommunication sessions can be initiated with a specific Fast Negotiationphase.
 12. The method of claim 7, wherein the protocol interacts withLocal resource management units during Pre-Negotiation, Multi-Stream QoSSynchronization and QoS Correlation, Fast Negotiation, renegotiation andResource Release phases.
 13. The method of claim 7, wherein a resourceadmission control and resource reservation are applied according to thefollowing order: at the initiator first, then at the responder(s), andfinally for the network.
 14. The method of claim 13, wherein localresource management tasks like real-time CPU scheduling, memorymanagement, and power management are addressed before applying admissioncontrol and resource reservation for the network.
 15. The method ofclaim 7, wherein for the resource release, first network resources, thenpeer and finally local resources are released.
 16. The method of claim7, wherein during runtime of a so established multimedia session, at anytime any component of any peer may request an adaptation, thuseventually triggering a renegotiation.
 17. The method of claim 7,further comprising: pre-negotiation of the type of E2ENP during ProtocolDiscovery phase, either by forcing peers to query a Directory Serverwhich may be implemented as a SIP registrar, or by having the peersannouncing such information.
 18. The method of claim 7, furthercomprising: pre-negotiation of a complete codec list during thepre-negotiation phase.
 19. The method of claim 7, further comprising:pre-negotiation of adaptation paths at stream level during thePre-Negotiation phase.
 20. The method of claim 7, further comprising:pre-negotiation of Adaptation Paths at stream aggregation level duringMulti-Stream QoS Synchronization and QoS Correlation phase.
 21. Themethod of claim 7, further comprising: indexing pre-negotiated QoSContracts and Capabilities for speeding up the Fast Negotiation phase.22. The method of claim 7, further comprising: indexing pre-negotiatedQoS Contracts and Capabilities for speeding up the renegotiation. 23.The method of claim 7, further comprising: handlinginstallation/deinstallation of Capabilities even at runtime, byexchanging asynchronous messages among peers for notifying such events.24. A peer, configured to exchange media streams with a plurality ofother peers in a network, comprising: a pre-negotiation unit configuredto negotiate a common set of a plurality of indexed Quality-of-Service(QoS) contracts and terminal capabilities between peers, wherein thepre- negotiating unit is further configured to forward indexed QoScontracts and terminal capabilities to at least one of the plurality ofother peers, and receive, from the at least one of the plurality ofother peers, a generated common subset of said indexed QoS contracts andterminal capabilities of the at least one of the plurality of otherpeers; a coordination unit configured to coordinate different phases ofa negotiation process of a distributed resource management process; andan establishment unit configured to negotiate QoS contracts on a perstream basis using the pre-negotiated QoS contracts and terminalcapabilities at the time of stream establishment, wherein in case of arenegotiation during streaming, one of the peers sends the index of oneof the pre-negotiated QoS contracts and terminal capabilities to theother peers.
 25. The peer of claim 24, wherein the coordination unit isfurther configured to command a Protocol Discovery, andtrigger/coordinate a: Pre-Negotiation, optional Multi-Stream QoSSynchronization and QoS Correlation, Fast-Negotiation with EconomyPrinciple, Renegotiation with Economy Principle, and ResourceReservation Release phase.
 26. The peer of claim 24, further comprising:a protocol discovery unit configured to advertise and query informationabout an end-to-end negotiation protocol to be used.
 27. The peer ofclaim 24, further comprising: a session protocol unit configured toallow the coordination unit to carry out different phases of thecoordination process with other peers.
 28. The peer of claim 24, furthercomprising: a plurality of interfaces configured to connect thecoordination unit to local resource management units.
 29. The peer ofclaim 24, further comprising: a plurality of interfaces configured toconnect the coordination unit to the protocol discovery unit.
 30. Thepeer of claim 24, further comprising: a plurality of interfacesconfigured to connect the coordination unit to the session protocolunit.
 31. The peer of claim 24, further comprising: a plurality ofinterfaces configured to connect the coordination unit to a FSM EngineUnit.
 32. The peer of claim 24, further comprising: a plurality ofinterfaces configured to connect the coordination unit to a networkresource reservation unit.
 33. A computer readable storage mediumcontaining program instructions, which when executed by a processor,cause the computer to perform a method for exchanging media streams witha plurality of other peers in a network, said method comprising:negotiating a common set of a plurality of indexed Quality-of-Service(QoS) contracts and terminal capabilities, wherein the pre-negotiatingincludes forwarding indexed QoS contracts and terminal capabilities toat least one of the plurality of other peers; and receiving, from the atleast one of the plurality of other peers, a generated common subset ofsaid indexed QoS contracts and terminal capabilities of the at least oneof the plurality of other peers; coordinating different phases of anegotiation process of a distributed resource management process; andnegotiating, at the peers, QoS contracts on a per stream basis using thepre-negotiated QoS contracts and terminal capabilities at the time ofstream establishment, wherein in case of a renegotiation duringstreaming, one of the peers sends the index of one of the pre-negotiatedQoS contracts and terminal capabilities to the other peers.